This invention relates to a video signal transmission system, and more particularly is suitably applied to a case in which video signals are transformed to high efficiency coded data and are then transmitted.
In the picture phone system and the video conference system, heretofore there has been proposed a video signal transmission system in which video signals which constitute moving pictures are high efficiency coded to provide intra coded frame data and inter coded frame data, and thereby the moving picture video signals are transmitted through transmission lines with relatively strict limits in transmission capacity (Japanese Patent Laid-open Publication No. 63 (1988)-1183).
More specifically, in a case where pictures PC1, PC2, PC3, . . . which constitute moving pictures at respective times t=t1, t2, t3, . . . are transmitted as shown in FIG. 1, the transmission efficiency is enhanced by compressing the picture data to be transmitted, using the fact that video signals are highly autocorrelated over time. In the intra coding, pictures PC1, PC2, PC3, . . . are compressed in such a manner that differences are determined by comparing pixel with a predetermined reference value. Thus, the amount of picture data transmitted for each picture PC1, PC2, PC3, . . . is compressed using the autocorrelation between pixel data in the same frame.
In the inter coding, picture data PC12, PC23, . . . which are differences between two successive pictures PC1 and PC2, PC2 and PC3, . . . , respectively, are determined as shown in FIG. 1, and are transmitted together with the intra coded picture data of the original picture PC1 at the time t=t1.
Thus, it is possible to send the pictures PC1, PC2, PC3, . . . in the format of high efficiency coded digital data which is markedly small in amount as compared to data in the case where all the picture data is sent.
The coding of such picture signals is achieved by a picture data generating system 1 having a configuration shown in FIG. 2.
The picture data generating system 1 performs processing operations, such as every other field dropping and every other field line dropping, on input video signals VD in a preprocessing circuit 2. Subsequently, the luminance signals and chrominance signals thereof are transformed to transmission unit block (referred to as macro block) data S11 which is 16 pixels (horizontal).times.16 pixels (vertical), and are then fed to a picture data coding circuit 3.
The picture data coding circuit 3 accepts predicted current frame data S12 constructed in a prediction coding circuit 4, and either determines the difference between the predicted current frame data S12 and the macro block data S11 to produce inter coded data (this is referred to as inter coding mode) or determines the difference between the macro block data S11 and the reference data to generate intra coded data. In either case, this data is fed to a transform coding circuit 5 as differential data S13.
The transform coding circuit 5 consists of a discrete cosine transform circuit and provides transform coded data S14, high efficiency coded by orthogonally transforming the differential data S13, to a quantizer 6, which sends quantized picture data S15.
The quantized picture data S15 thus obtained from the quantizer 6 is high efficiency coded again in a retransform coding circuit 7 which includes a variable length coding circuit and is then fed a s transmission picture data S16 to a transmission buffer memory 8.
Moreover, the quantized picture data S15 undergoes inverse quantization and inverse transform coding operations in the prediction coding circuit 4, so that the quantized picture data S15 is decoded to differential data. Then, the predicted previous frame data is modified by the differential data, and thereby new predicted previous frame data is stored in the prediction coding circuit 4. Furthermore, the new predicted previous frame data which is stored is motion compensated by motion detection data, which has been constructed on the basis of the macro block data S11, to produce predicted current frame data for feeding to the picture data coding circuit 3. Thus, the difference between the macro block data S11 of a frame (the current frame) which is to be transmitted and the predicted current frame data S12 is determined and supplied as the differential data S13.
When the motion pictures mentioned above in referring to FIG. 1 are sent with the construction of FIG. 2, the picture data of the picture PC1 is firstly provided as macro block data S11 at the time t1 in FIG. 1. In this case, the picture data coding circuit 3 is placed in the intra coding mode, and provides intra coded differential data S13 to the transform coding circuit 5. In this manner transmission picture data S16 is fed to the transmission buffer memory 8 through the quantizer 6 and the retransform coding circuit 7.
On the other hand, the quantized picture data S15 obtained at the output of the quantizer 6 is prediction coded in the prediction coding circuit 4, and thereby predicted previous frame data which represents the transmission picture data S16 sent to the transmission buffer memory 8 is held in the previous frame memory. When macro block data S11 which represents the picture PC2 at time t2 is fed to the picture data coding circuit 3, the predicted previous frame data is motion compensated to produce the predicted current frame data S12, which is provided to the picture data coding circuit 3.
Thus, the picture data coding circuit 3 provides differential data S13 inter coded to the transform coding circuit 5 at the time t=t2. In this manner, the differential data which represents changes in the picture between the frames is sent as transmission picture data S16 to the transmission buffer memory 8 while the quantized picture data S15 thereof is provided to the prediction coding circuit 4 to thereby construct and store predicted previous frame data in the prediction coding circuit 4.
Only the differential data representing changes in the picture between the frames is sequentially transmitted to the transmission buffer memory 8 by repeating similar operations during the inter coding operation of the picture data coding circuit 3.
The transmission buffer memory 8 accumulates transmission picture data S16 which has been received in this manner, and successively sends the accumulated transmission picture data S16 as transmission data D.sub.TRANS to a transmission line 9 at a data transmission rate which is determined according to the transmission capacity of the transmission line 9.
Concurrently, the transmission buffer memory 8 detects the amount of residual data contained therein, in order to feed back residual amount data S17, which changes according to the amount of the residual data, to the quantizer 6. In this manner the quantization step size is controlled according to the residual amount data S17 in order to regulate the amount of data generated as the transmission picture data S16, so that an appropriate amount of residual data (so as not to produce overflow or underflow) is kept in the transmission buffer memory 8.
When the amount of residual data in the transmission buffer memory 8 increases to a predetermined upper limit, rough quantization is carried out in the quantizer 6 by enlarging the step size of the quantization step STPS (FIG. 3) of the quantizer 6, so that the amount of the transmission picture data S16 generated is reduced.
On the contrary, when the amount of residual data in the transmission buffer memory 8 decreases to a predetermined lower limit, the step size of the quantization step STPS of the quantizer 6 is reduced to a smaller value, and thus the amount of the transmission picture data S16 generated is increased by performing fine quantization in the quantizer 6.
More specifically, in the transmission system, an evaluation function is performed stepwise by a motion vector detection circuit with reference to a picture of a predetermined frame (hereinafter referred to as the reference frame).
Moreover, in the transmission system the picture of the reference frame is shifted by the motion vector to produce a comparison reference picture, and then differential data between that comparison reference picture and the picture to be transmitted is calculated. The differential data is transmitted together with the motion vector.
In the receiving system, the reference frame picture previously transmitted is shifted by the motion vector sent, and then the differential data sent is added to reconstruct the original picture.
In the conventional picture data generating system 1, the transmission buffer memory 8 is, as described, provided as a means for transmitting significant picture information while the data transmission rate of the transmission data D.sub.TRANS is limited according to the transmission capacity of the transmission line 9, and thereby picture data is accumulated in the transmission buffer memory so that the picture data of the transmission capacity of the transmission line 9 may be sent without excess or deficiency. However, it is possible in practice that the transmission buffer memory 8 overflows when the amount of picture data generated in the picture data generating system 1 becomes extremely large.
Moreover, in the picture data generating system 1 of such a construction, there is a problem in that the transmission buffer memory overflows due to an increase of the amount of coded transmission data generated when moving pictures to be transmitted are moving rapidly or when the contents thereof significantly changes, for example, as in a change of scene.
As one technique to solve the problem, there has been suggested a frame dropping technique in which subsequent frames are not coded or transmitted when the amount of transmission data generated is excessive.
In a case where rapidly moving pictures are sent for a relatively long time, however, frame dropping operations are repeated often since the amount of the transmission data generated continues to increase. Thus, there is a problem in that visual smoothness of movement of the reproduced moving pictures is lost, and hence this technique is not sufficient for sending moving pictures.
Furthermore, in this type of video signal transmission system using inter coding, one frame of a picture is divided into a plurality of regions (hereinafter referred to as macro blocks) and a motion vector is determined for each macro block.
More specifically, within a predetermined motion vector detection range the picture of a reference frame is sequentially moved relative to each macro block, and thereby the shift position (which provides the motion vector) for which the differential data is minimized determined.
The picture of the reference frame is shifted to the determined position for the region which corresponds to the macro block, and thereby picture data of a comparison reference for the differential data is produced.
By repeating this procedure, the differential data is determined for each macro block, and is then sent after discrete cosine transformation, quantization, and variable length coding operations.
Thus, in this type of video signal transmission system, the picture data is coded in such a way that boundaries between macro blocks become obtrusive when a motion vector is detected for each macro block.
To make these boundaries unobtrusive, there has been considered a technique in which in determining differential data a high frequency zone of picture data for comparison reference is suppressed by means of a filter circuit as disclosed in Japanese Patent Laid-open Publication 60(1985)-206,318, for example.
For making the boundaries unobtrusive by suppressing a high frequency zone of picture data for comparison reference, it is necessary to change the characteristic of the filter according to the position of the picture data, and there is hence a problem in that practical filter circuits are needed to perform this processing in real time.
Furthermore, a picture of the reference frame is necessary for inter coding processing to decode the original picture data.
For this reason, in this type of video signal transmission system the immediately previous frame, for example, ma be selected as the reference frame, and intra coded picture data may be sent in place of inter coded differential data for each such predetermined frame.
Intra coding is a process which codes picture data at a high efficiency using the correlation of data within the frame and is inferior in efficiency to the inter coding process. However, with intra coding the original picture can be reconstructed from only the picture data sent.
Accordingly, the original picture data can be reconstructed on the basis of the differential data transmitted if the immediately previous frame is intra coded, and if with that frame selected as the reference frame, picture data of the subsequent frame is inter coded.
Moreover, on the basis of the picture data reconstructed in this manner picture data of subsequent frames can be also reconstructed.
In this type of video signal transmission system, the transmission line is connected after the coding of picture data is commenced.
Thus, inter coding may be performed and repeated just after the line is connected. In this case, the correct picture cannot be reconstructed at the receiving end until intra coded picture data is sent.
More specifically, in the case where intra coded picture data is sent every 128 frames, for example, the speaker at the receiving end must talk, while monitoring an unnatural picture displayed for about 13 seconds maximum when the frame frequency of the picture data is 10 Hz. Thus, there is a problem with the practical usability of the transmission system as described.